Synchronization of signals

ABSTRACT

A method and system for synchronizing signals. First and second signals are sent (compressed or uncompressed) from a source to a receiving apparatus of a receiving system. The first signal has content of a first modality (e.g., audio) and the second signal has content of a second modality (e.g., video). The first and second signals are to be displayed on a display apparatus of the receiving system. The first and second signals have been time-synchronized at the source. If the first and second signals are not synchronized in time when received at the receiving apparatus, then the first and second signals may be time-synchronized at the receiving apparatus either manually or through timestamping of both signals at the source, in conjunction with use of a real-time clock at the receiving apparatus.

Cross Reference to Related Cases

Applicants claim the benefit of Provisional Application Ser. No.60/430,919, filed Dec. 4, 2002.

The present invention relates to a method and system for synchronizingsignals.

A practical technology for generally synchronizing audio and videosignals transmitted from a sender to a receiver does not exist. Thusthere is a need for a method and system to generally synchronize audioand video signals transmitted from a sender to a receiver.

The present invention provides a method for synchronizing signals,comprising:

-   receiving, from a source, a first signal and a second signal by a    receiving apparatus of a receiving system, the first and second    signals to be displayed on a display apparatus of the receiving    system, the first and second signals having been time-synchronized    at the source, the first signal having content of a first modality,    the second signal having content of a second modality;-   displaying on the display the first and second signals, said    displayed first and second signals being accessible to a user; and-   manually reducing, by the user, the time rate of displaying one of    the first signal and the second signal, said manually reducing being    directed to time-synchronizing said displaying of the first and    second signals on the display apparatus.

The present invention provides a system for synchronizing signals,comprising:

-   receiving means for receiving, from a source, a first signal and a    second signal by a receiving apparatus of a receiving system, the    first and second signals to be displayed on a display apparatus of    the receiving system, the first and second signals having been    time-synchronized at the source, the first signal having content of    a first modality, the second signal having content of a second    modality;-   display means for displaying on the display the first and second    signals, said displayed first and second signals being accessible to    a user; and-   manual reducing means for manually reducing, by the user, the time    rate of displaying one of the first signal and the second signal,    said manually reducing being directed to time-synchronizing said    displaying of the first and second signals on the display apparatus.

The present invention provides a method for synchronizing signals,comprising:

-   receiving, from a source, a first signal and a second signal by a    receiving apparatus of a receiving system, the first and second    signals to be synchronously displayed on a display apparatus of the    receiving system, the first and second signals having been    time-synchronized at the source, the first signal having content of    a first modality and a first plurality of time stamps originating    from the source, the second signal having content of a second    modality and a second plurality of time stamps originating from the    source, the second plurality of time stamps being synchronized with    the first plurality of time stamps;-   determining at a plurality of times on a real-time clock CR at the    receiving system whether the first and second signals are    time-synchronized relative to the clock CR, said determining being    based on analyzing the first and second plurality of time stamps in    relation to the clock CR; and-   reducing the time rate of displaying one of the first signal and the    second signal when said determining determines that the first and    second signals are not time-synchronized relative to the clock CR    such that the one is time advanced relative to the remaining other    of the first signal and the second signal, said reducing being    directed to time-synchronizing said displaying of the first and    second signals on the display apparatus.

The present invention provides a system for synchronizing signals,comprising:

-   receiving means for receiving, from a source, a first signal and a    second signal by a receiving apparatus of a receiving system, the    first and second signals to be synchronously displayed on a display    apparatus of the receiving system, the first and second signals    having been time-synchronized at the source, the first signal having    content of a first modality and a first plurality of timestamps    originating from the source, the second signal having content of a    second modality and a second plurality of timestamps originating    from the source, the second plurality of time stamps being    synchronized with the first plurality of time stamps;-   determining means for determining at a plurality of times on a    real-time clock CR at the receiving system whether the first and    second signals are time-synchronized relative to the clock CR, said    determining being based on analyzing the first and second plurality    of timestamps in relation to the clock CR; and-   reducing means for reducing the time rate of displaying one of the    first signal and the second signal when said determining determines    that the first and second signals are not time-synchronized relative    to the clock CR such that the one is time advanced relative to the    remaining other of the first signal and the second signal, said    reducing being directed to time-synchronizing said displaying of the    first and second signals on the display apparatus.

The present invention advantageously provides a method and system togenerally synchronize audio and video signals transmitted from a senderto a receiver.

FIG. 1 is a block diagram of a system for manually synchronizingsignals, in accordance with embodiments of the present invention.

FIG. 2 is a flow chart describing manual synchronisation of signals inconjunction with the manual system of FIG. 1, in accordance withembodiments of the present invention.

FIG. 3 is a block diagram of an automated system for synchronizingsignals using timestamps, in accordance with embodiments of the presentinvention.

FIG. 4 is a flow chart describing synchronisation of signals usingtimestamps in conjunction with the automated system of FIG. 3, inaccordance with embodiments of the present invention.

FIG. 5 illustrates definitions of parameters appearing in the flow chartof FIG. 4, in accordance with embodiments of the present invention.

FIG. 6 is a table describing timestamp content for uncompressed andcompressed signals in conjunction with the automated system of FIGS. 3-4in accordance with embodiments of the present invention.

FIG. 7 is a table listing signal modality combinations, in accordancewith embodiments of the present invention.

The present invention discloses a manual method and system, as well asan automated method and system, for synchronizing two or more signalstransmitted from a source (i.e., a sender) to a receiver on separatechannels. The modality of a signal is indicative of the type of contentcarried by the signal. Examples of modalities include a video modality,an audio modality, and a text modality, and the signals having thesemodalities are called video signals, audio signals, and text signals,which carry video content, audio content, and text content,respectively. FIG. 7 is a table listing signal modality combinations aspairs of modality types wherein the different modalities of such a pairmay characterize signals being transmitted from the source to thereceiver on separate channels, in accordance with embodiments of thepresent invention. The signal modality combinations in FIG. 7 include: avideo modality and an audio modality, a video modality and a textmodality, and an audio modality and a text modality. Although theembodiments of the present invention are described infra in terms ofsynchronizing an audio signal with a video signal, or vice versa, thescope of the present invention includes synchronizing signals having anymodalities. As indicated supra, the signals transmitted from the sourcemay have different modalities (e.g., an audio modality for a firstsignal transmitted from the source and a video modality for a secondsignal transmitted from the source). Nonetheless, the scope of thepresent invention includes the case in which the signals transmittedfrom the source have the same modality (e.g., a video modality for afirst signal transmitted from the source and a video modality for asecond signal transmitted from the source; an audio modality for a firstsignal transmitted from the source and an audio modality for a secondsignal transmitted from the source).

The signals sent to a receiver from the source are assumed to betime-synchronized at the source in accordance; e.g., with a real-timeclock at the source or studio “genlock” signals at the source. Saidsignals, after being sent from the source and before being received atthe receiver, may be delayed in transmission due to, inter alia, processdelay. For example, time delay may be caused by a signal going throughmany multiplex (broadcast) points; at each such broadcast point thesignal may be edited, or there may be other process delay (e.g.,processing due to inserting special effects into the video). Thus, thetransmission path itself may change the timing of the audio and videosignals differentially, since the audio and video signals travel alongdifferent transmission paths and are subject to different processingsteps. As a result, the audio and video signals may not be timesynchronized when the audio and video signals arrive at the receiver.The scope of the present invention includes cases in which the audio andvideo signals are not time synchronized when the audio and video signalsarrive at the receiver, for any reason. As a first example, the audioand video signals may not be time synchronized when arriving at thereceiver, because the first signal and the second signal are received atthe receiver on separate channels. As a second example, the audio andvideo signals may not be time synchronized when arriving at thereceiver, because the first signal and the second signal are received atthe receiver as not multiplexed with each other. As a third example, theaudio and video signals may not be time synchronized when arriving atthe receiver, because the first signal and the second signal arereceived at the receiver as multiplexed but not time-synchronized witheach other. The present invention is directed to synchronizing the audioand video signals at the receiver when the audio and video arrive out ofsynch at the receiver.

FIGS. 1-2 relate to the manual method and system of the presentinvention for synchronizing an audio and video signal with each other.FIGS. 3-6 relate to the automated method and system of the presentinvention for synchronizing an audio and video signal with each other.The audio and video signals are assumed to be content related, whichmeans that the audio and video signals are components of a commonprogram or other presentation such as a movie or television program, sothat corresponding portions of the audio and video signals are intendedto be played back together as the common program or other presentationis being played back. Definitionally, the audio and video signals (orany plurality of signals having different modalities, or the samemodality, for their content) are said to be synchronized if they aresynchronized in time, which means that when the audio and video signalsare played back, in accordance with the common program or otherpresentation being played back, said corresponding portions of the audioand video signals may be played back simultaneously. Thus the words“synchronize” and “time-synchronize” (and variants thereof) have thesame meaning herein.

FIG. 1 is a block diagram of a system 10 for manually synchronizingsignals, in accordance with embodiments of the present invention. Thesystem 10 is adapted to receive an audio signal 21 and a video signal 22from a source 20. The source 20 functions as a sender of the audiosignal 21 on audio channel 31, and a video signal 22 on video channel32, to the system 10. The audio signal 21 and the video signal 22 areeach either in an uncompressed format or in a compressed format, as sentby the source 20 and as received by the system 10. The video signal 22is either in an uncompressed format or in a compressed format, as sentby the source 20 and as received by the system 10. The source 10includes a real-time clock 24 which may be used to synchronize 26 theaudio signal 21 with the video signal 22, although said synchronizationbetween the audio signal 21 and the video signal 22 may alternatively beachieved by other methods such as, inter alia, utilizing a “genlock”signal at the source 20. If either or both of the audio signal 21 andthe video signal 22 are sent to the system 10 in a compressed format,the synchronization 26 between the audio signal 21 and the video signal22 is effectuated prior to said compression.

The system 10 includes a receiving apparatus 30, a display apparatus 40and a user apparatus 44. The receiving apparatus 30 includes adecompressor 35, a delay compensation circuit 33, and a delay line 34.The delay compensation circuit 33 is coupled to the delay line 34. Asexplained supra, the audio signal 21 and the video signal 22 may arriveat the receiving apparatus 30 out of time synchronization with eachother.

The display apparatus 40, which may include a control 42, may be anytype of apparatus known by one of ordinary skill in the art as beingcapable of displaying the pertinent signals. For example, the displayapparatus 40 may comprise a speaker for displaying the audio signal 21.As another example, the display apparatus 40 may comprise a televisionscreen or computer monitor for displaying the video signal 22. Thedisplay apparatus 40 is coupled to the delay line 34 of the receivingapparatus 30.

The user apparatus 44 comprises user input 45 and may additionallycomprise a wireless control 47. At least one of the control 42 and thewireless control 47 must be present.

Thus, the user input 45 is coupled to at least one of the control 42 andthe wireless control 47. The user input 45 may transfer input from auser to the delay compensation circuit 33 through the control 42, thewireless control 47, or both.

After being received by the receiving apparatus 30, the audio signal 21is decompressed by the decompressor 35 if the audio signal 21 was sentby the source 20 in compressed format, and the video signal 22 isdecompressed by the decompressor 35 if the video signal 22 was sent bythe source 20 in compressed format. Next, the audio signal 21 and thevideo signal 22 enter the delay line 34 where one or both of saidsignals are subject to being delayed or slowed down under control of thedelay compensation circuit 33. The delay line 34 may include one or morebuffers for storing the audio signal 21 and the video signal 22

The display apparatus 40 is adapted to display the audio signal 21 andthe video signal 22. The audio signal 21 and the video signal 22 beingdisplayed on the display apparatus 40 are accessible to a user; i.e.,the audio signal 21 may be heard by the user and the and the videosignal 22 may be seen by the user. The user may manually reduce the timerate of displaying the one signal (i.e., the “advanced” signal) of theaudio signal 21 and the video signal 22 that is ahead of the otherremaining signal (i.e., the “retarded” signal), so as totime-synchronize the displaying of the audio signal 21 and the videosignal 22 on the display apparatus 40. This synchronization isaccomplished by transmitting user input 45 from the user to the delaycompensation circuit 33 via the control 41 on the display apparatus 40or via the wireless control 47. The user input 45 is comprises a delaydirective from a user to the delay compensation circuit 33. The control41 and the wireless control 47 each comprise any input device ormechanism (e.g., a keyboard, mouse, turnable knob, etc) capable oftransmitting the user input 45 to the delay compensation circuit 33.

The time rate of displaying the advanced signal may be reducedcontinuously via the delay compensation circuit 33 such as by reducingthe transmission rate or display rate of the advanced signal. Forexample, the user may turn a knob on the control 41 or wireless control47 such that said time rate of displaying increases as the knob isturned clockwise and said time rate of displaying decreases as the knobis turned counterclockwise. The time rate of displaying the advancedsignal may alternatively be reduced discontinuously by introducing atime delay gap in the advanced signal via the delay compensation circuit33. The time delay gap is an interval of time during which the advancedsignal is held in suspension and is not being further transmitted. Forexample, the user may press a button that acts as a toggle switch:pressing the button a first time stops transmission of the advancedsignal and pressing the button a second time resumes transmission of theadvanced signal. The advanced signal is frozen within the one or morebuffers of the delay line 34 for the period of the time delay gap.

FIG. 2 is a flow chart comprising steps 51-56 which describe manualsynchronization of signals in conjunction with the manual system 10 ofFIG. 1, in accordance with embodiments of the present invention. In step51, the user samples the audio signal 21 (S1) and the video signal 22(S2) by listening to audio signal 21 and viewing the video signal 22.Through said sampling, the user determines in step 52 whether thesignals S1 and S2 are in synch (i.e., in synchronization with eachother). If the signals S1 and S2 are in synch, then the user continuesin step 51 to sample the signals S1 and S2. If and when the userdetermines that the signals S1 and S2 are not in synch, then the userdetermines in step 53 that one of the signals, namely S1 for example, istime-lagging the other signal S2. In step 54, the user manually reduces(continuously or discontinuously as described supra) the rate ofdisplaying the signal S2 until the user is satisfied that the signals S1and S2 are in synch. In step 55, the user decides whether to continue orstop sampling the signals S1 and S2. As a result, the user may stopsampling the signals S1 and S2 (step 56) or return to step 51 tocontinue to sample the signals S1 and S2.

FIG. 3 is a block diagram of an automated system 60 for synchronizingsignals, in accordance with embodiments of the present invention. Thesystem 60 is adapted to receive an audio signal 71 and a video signal 72from a source 70. The source 70 functions as a sender of the audiosignal 71 on audio channel 81, and as a sender of the video signal 72 onvideo channel 82, to the system 60. The audio signal 71 and the videosignal 72 are each in either in an uncompressed format or in acompressed format, as sent by the source 70 and as received by thesystem 60. The video signal 72 is either in an uncompressed format or ina compressed format, as sent by the source 70 and as received by thesystem 60. The source 70 includes a real-time clock 74 which maysynchronize 76 the audio signal 71 with the video signal 72, althoughsaid synchronization between the audio signal 71 and the video signal 72may alternatively be achieved by other methods such as, inter alia,utilizing a “genlock” signal at the source 70. If either or both of theaudio signal 71 and the video signal 72 are sent to the system 60 in acompressed format, the synchronization between the audio signal 71 andthe video signal 72 is effectuated prior to said compression.

The timestamp inserter 76 inserts a plurality of timestamps into each ofthe audio signal 71 and the video signal 72. These timestamps will besubsequently utilized by the receiving apparatus 80 to synchronize theaudio signal 71 and the video signal 72 if said signals arrived at thereceiving apparatus 80 out of synch with each other. The timestampcontent to be inserted in a given signal (i.e., the audio signal 71 orthe video signal 72) depends on whether the given signal to be sent bythe source to the receiving apparatus 80 is uncompressed or compressed.If the given signal is uncompressed, then the timestamp includes a time(e.g., a time read from the real-time clock 74) that is inserted intothe given signal as the given signal is being transmitted to thereceiving apparatus 80, or alternatively when the given signal is beingpropagated at the rate of transmission at which the given signal isintended to be subsequently played back. The timestamp may be stored ina location in the given uncompressed signal from which the timestamp canbe later retrieved. For example, if the given uncompressed signal is avideo signal, then the timestamp can be stored in, inter alia, blankingintervals between frames. As another example, if the given uncompressedsignal is a audio signal, then the timestamp can be stored as awatermark in the audio signal. The watermark may include audiofrequencies and/or audio amplitudes that lie outside the range of humanhearing.

If the given signal is compressed, then the timestamp includes: 1) atime (e.g., as read from the real-time clock 74), wherein the timecorresponds to a location (L) in an uncompressed signal which wascompressed to form the given signal; and 2) an identification of saidlocation (L) in the uncompressed signal. The location (L) needs to be inthe timestamp because there is no precise correlation between the bitsof a compressed signal and locations in the corresponding uncompressedsignal. Thus the location (L) compensates for said lack of precisecorrelation. FIG. 1 indicates that the timestamp inserter 76 comprisesthe compression functionality if compression is relevant. This meansthat the timestamp may be inserted into the given signal whilecompression is occurring. Alternatively, the given signal may becompressed first followed later by insertion of the timestamp into thecompressed signal. The timestamp may be stored in any location in thegiven compressed signal from which the timestamp can be later retrieved.For example, the timestamp may be inserted between two logicalcompression units of the compressed signal. FIG. 6 is a tablesummarizing the timestamp contents for the uncompressed signal formatand the compressed signal format.

The system 60 includes a receiving apparatus 80 and a display apparatus90. Although not shown, the system 60 may also include a user apparatusanalogous to the user apparatus shown in FIG. 1 and described supra. Ifthe system 60 includes such a user apparatus, then the system 60 wouldhave both manual capabilities and the automated capabilities describedinfra.

As explained supra, the audio signal 71 and the video signal 72 mayarrive at the receiving apparatus 80 out of time synchronization witheach other. The receiving apparatus 80 includes a real-time clock 86, atimestamp extractor 85, a delay compensation controller 83, and a delayline 84. The delay compensation controller 83 is coupled to thereal-time clock 86, the delay line 84, and the timestamp extractor 85.The timestamp extractor 85 is coupled to the delay line 84. As indicatedin FIG. 3, the timestamp extractor 85 also includes decompression meansfor decompressing whichever of the audio signal 71 and the video signal72 has been received by the receiving apparatus 80 in compressed format.

Regardless of whether the audio signal 71 and the video signal 72 arereceived by the receiving apparatus 80 in uncompressed format or incompressed format, the timestamp extractor 85 extracts the timestampsfrom within the audio signal 71 and the video signal 72. The timestampshad been independently inserted into the audio signal 71 and the videosignal 72 by the timestamp inserter 76, and the timestamps areindependently extracted from the audio signal 71 and the video signal 72by the timestamp extractor 85.

The display apparatus 90, (which may also include a control such as thecontrol 42 shown in FIG. 1 and described supra) may be any type ofapparatus known by one of ordinary skill in the art as being capable ofdisplaying the pertinent signals. For example, the display apparatus 90may comprise a speaker for displaying the audio signal 71. As anotherexample, the display apparatus 90 may comprise a television screen orcomputer monitor for displaying the video signal 72. The displayapparatus 90 is coupled to the delay line 84 of the receiving apparatus80.

After having each timestamp extracted by the timestamp extractor 85, theaudio signal 71 and the video signal 72 are transmitted into the delayline 84 where one or both of said signals are subject to being delayedor slowed down under control of the delay compensation controller 83.The delay line 84 may include one or more buffers for storing the audiosignal 71 and the video signal 72. The audio signal 71 and the videosignal 72 are transmitted from the delay line 84 into the displayapparatus 90. The display apparatus 90 is adapted to display the audiosignal 71 and the video signal 72.

The delay compensation controller 83 may effectuate a reduction in thetime rate of displaying the one signal (i.e., the “advanced” signal) ofthe audio signal 71 and the video signal 72 that is ahead of the otherremaining signal (i.e., the “retarded” signal), so as totime-synchronize the displaying of the audio signal 71 and the videosignal 72 on the display apparatus 90. This synchronization isaccomplished in accordance with FIG. 4, which is a flow chart describingsynchronization of signals using timestamps, in accordance withembodiments of the present invention.

The flow chart of FIG. 4 comprises steps 91-96. As stated supra, theaudio signal 71 and the video signal 72 have their timestamps extractedindependently after the audio signal 71 and the video signal 72 enterthe receiving apparatus 80 and are decompressed (if received incompressed format). For simplicity, let S1 represent the audio signal 71and let S2 represent the video signal 72. In step 91, the delaycompensation controller 83 samples a timestamp from signal S1 or atimestamp from signal S2. Let the symbol T1 denote a normalized time ofthe timestamp from signal S1, wherein the normalized time T1 is theactual time of the timestamp plus an additive constant C1 thatnormalizes the timestamps of the signal S1 to the time frame of thereal-time clock 86. For example, if the first timestamp existing in thesignal S1 is “10:00 AM” and is processed by the delay compensationcontroller 83 when the delay compensation controller 83 reads the timeon the real-time clock 86 as “9:30 AM”, then the additive constant C1 of30 minutes of time is added to every timestamp value extracted from thesignal S1 for normalization purposes. The parameter T1 is the timestampvalue of time, relating to the signal S1, after said normalization. Saidnormalization makes it possible to make meaningful comparisons betweenthe timestamps of the signal S1 with the real-time clock 86 in order todetect whether the various timestamps of the signal S1 are out of synchwith the real-time clock 86. Similarly, let the symbol T2 denote anormalized time of the timestamp from signal S2, wherein the normalizedtime T2 is the actual time of the timestamp plus an additive constant C2that normalizes the timestamps of the signal S2 to the time frame of thereal-time clock 86. The additive constants C1 and C2 are independent ofeach other and may be positive, negative, or zero.

Step 92 compares the normalized timestamp time T1 of the signal S1 withthe corresponding time TC1 on the clock 86. If T1 is in synch with theclock 86, then T1 equals TC1. Accordingly, step 92 calculatesΔT1=TC1−T1, which represents an amount of time by which T1 is out ofsynch with the clock 86. If ΔT1>0, then the signal S1 lags the clock 86at time T1. If ΔT1<0, then the signal S1 leads the clock 86 at time T1.If ΔT1=0, then the signal S1 is in synch with the clock 86 at time T1.

Step 93 compares the normalized timestamp time T2 of the signal S2 withthe corresponding time TC2 on the clock 86. If T2 is in synch with theclock 86, then T2 equals TC2. Accordingly, step 92 calculatesΔT2=TC2−T2, which represents an amount of time by which T2 is out ofsynch with the clock 86. If ΔT2>0, then the signal S2 lags the clock 86at time T2. If ΔT2<0, then the signal S2 leads the clock 86 at time T2.If ΔT2=0, then the signal S2 is in synch with the clock 86 at time T2.FIG. 5 is a time line illustrating the definitions of various parametersT1, TC1, ΔT1, T2, TC2, and ΔT2.

Step 94 compares ΔT2 with ΔT1 to determine whether signal S2 is in synchwith signal S1. If ΔT2=ΔT1, then signal S2 is in synch with signal S1and processing returns to step 91. If ΔT2>ΔT1, then signal S2 is out ofsynch with signal S1 such that signal S2 lags signal S1 and the rate ofdisplaying signal S1 is reduced in step 95 by delay compensationcontroller 83 followed by returning to step 91. If ΔT2<ΔT1, then signalS2 is out of synch with signal S1 such that signal S1 lags signal S2 andthe rate of displaying signal S2 is reduced in step 95 by delaycompensation controller 83 followed by returning to step 91.

Step 94 may be performed every time a new value of ΔT1 or ΔT2 iscalculated in step 92 or step 93. Alternatively, step 94 may beperformed selectively and not following every calculation of ΔT1 or ΔT2.The time retardation of S1 in step 95 or of S2 in step 96 may be equalto ΔT2−ΔT1 or ΔT1−ΔT2, respectively, which neglects the fact that thecomparison between T1 and T2 may be made at different times on thereal-time clock 86; i.e., when TC1 ¹ TC2. Improved accuracy may beobtained by using interpolation or extrapolation on T1 to adjust ΔT1 tocompensate for TC1 differing from TC2, or by using interpolation orextrapolation on T2 to adjust ΔT2 to compensate for TC2 differing fromTC1. The scope of the present invention includes any numerical method ofeffectuating a time-retardation of signal S1 or of signal S2 as would beapparent to one of ordinary skill in the art.

The time rate of displaying the advanced signal may be reducedcontinuously such as by having the delay compensation controller 83reduce the transmission rate or display rate of the advanced signal.(i.e., the one signal of the audio signal 71 and the video signal 72that is ahead of the other remaining signal). The time rate ofdisplaying the advanced signal may alternatively be reduced by havingthe delay compensation controller 83 introduce a time delay gap in theadvanced signal via freezing the advanced signal in the one or morebuffers of the delay line 84 for the period of the time delay gap. Thetime delay gap is an interval of time during which the advanced signalis held in suspension and is not being further transmitted.

The present invention is generally applicable to any situation in whichdifferent signals transmitted from a source to a receiver are timesynchronized at the receiver. As an example, the present invention maybe applied to a video game in which the audio and video signals for thevideo game may arrive at receiver (where one or more players of the gameare located) out of time synchronization with respect to each other. Theout-of-synch signal conditions may occur periodically so that thecorrective action disclosed by the present invention (i.e., reducing therate of displaying the audio signal or the video signal) may be appliedconditionally. Depending on the nature of the video game, it may even bedesirable at times not to take corrective action according to thepresent invention when out-of-synch signal conditions occur under somespecial circumstances relating to the video game.

While embodiments of the present invention have been described hereinfor purposes of illustration, many modifications and changes will becomeapparent to those skilled in the art. Accordingly, the appended claimsare intended to encompass all such modifications and changes as fallwithin the true spirit and scope of this invention

1. A method for synchronizing signals, comprising: receiving, from asource, a first signal and a second signal by a receiving apparatus of areceiving system, the first and second signals to be synchronouslydisplayed on a display apparatus of the receiving system, the first andsecond signals having been time-synchronized at the source, the firstsignal having content of a first modality and a first plurality of timestamps originating from the source, the second signal having content ofa second modality and a second plurality of time stamps originating fromthe source, the second plurality of time stamps being synchronized withthe first plurality of time stamps; determining at a plurality of timeson a real-time clock at the receiving system whether the first andsecond signals are time-synchronized relative to the clock, saiddetermining being based on analyzing the first and second plurality oftime stamps in relation to the clock; and reducing the time rate ofdisplaying one of the first signal and the second signal when saiddetermining determines that the first and second signals are nottime-synchronized relative to the clock such that the one is timeadvanced relative to the remaining other of the first signal and thesecond signal, said reducing being directed to time-synchronizing saiddisplaying of the first and second signals on the display apparatus. 2.The method of claim 1, wherein the first signal is in an uncompressedformat as received by the receiving apparatus, wherein each timestamp ofthe first plurality of timestamps includes a time corresponding to alocation in the first signal at which said timestamp of the firstplurality of timestamps is positioned, wherein the second signal is inan uncompressed format as received by the receiving apparatus, andwherein each timestamp of the second plurality of timestamps includes atime corresponding to a location in the second signal at which saidtimestamp of the second plurality of timestamps is positioned.
 3. Themethod of claim 1, wherein the first signal is in a compressed format asreceived by the receiving apparatus, wherein each timestamp of the firstplurality of timestamps includes a time corresponding to a location in afirst uncompressed signal which was compressed at the source to form thefirst signal, wherein said timestamp of the first plurality oftimestamps further includes an identification of said location in thefirst uncompressed signal, wherein the second signal is in a compressedformat as received by the receiving apparatus, wherein each timestamp ofthe second plurality of timestamps includes a time corresponding to alocation in a second uncompressed signal which was compressed at thesource to form the second signal, and wherein said timestamp of thesecond plurality of timestamps further includes an identification ofsaid location in the second uncompressed signal.
 4. The method of claim1, wherein the first signal is in an uncompressed format as received bythe receiving apparatus, wherein each timestamp of the first pluralityof timestamps includes corresponding to a location in the first signalat which said timestamp of the first plurality of timestamps ispositioned, wherein the second signal is in a compressed format asreceived by the receiving apparatus, wherein each timestamp of thesecond plurality of timestamps includes a time corresponding to alocation in an uncompressed signal which was compressed at the source toform the second signal, and wherein said timestamp of the secondplurality of timestamps further includes an identification of saidlocation in the uncompressed signal.
 5. The method of claim 1, whereinsaid reducing is effectuated by a delay compensation controller of thereceiving apparatus.
 6. The method of claim 1, wherein said reducingcomprises introducing a time delay gap in the displaying of said one ofthe first signal and the second signal.
 7. The method of claim 1,wherein said reducing does not comprise introducing a time delay gap inthe displaying of said one of the first signal and the second signal. 8.The method of claim 1, wherein the first modality differs from thesecond modality.
 9. The method of claim 8, wherein the first modality isa video modality, and wherein the second modality is an audio modality.10. The method of claim 8, wherein the first modality is a videomodality, and wherein the second modality is an text modality.
 11. Themethod of claim 8, wherein the first modality is a text modality, andwherein the second modality is an audio modality.
 12. The method ofclaim 1, wherein said receiving the first signal and the second signalcomprises receiving the first signal and the second signal on separatechannels.
 13. The method of claim 1, wherein said receiving the firstsignal and the second signal comprises receiving the first signal andthe second signal as not multiplexed with each other.
 14. The method ofclaim 1, wherein said receiving the first signal and the second signalcomprises receiving the first signal and the second signal asmultiplexed but not time-synchronized with each other.
 15. A system forsynchronizing signals, comprising: receiving means for receiving, from asource, a first signal and a second signal by a receiving apparatus of areceiving system, the first and second signals to be synchronouslydisplayed on a display apparatus of the receiving system, the first andsecond signals having been time-synchronized at the source, the firstsignal having content of a first modality and a first plurality oftimestamps originating from the source, the second signal having contentof a second modality and a second plurality of timestamps originatingfrom the source, the second plurality of time stamps being synchronizedwith the first plurality of time stamps; determining means fordetermining at a plurality of times on a real-time clock C_(R) at thereceiving system whether the first and second signals aretime-synchronized relative to the clock C_(R), said determining beingbased on analyzing the first and second plurality of timestamps inrelation to the clock C_(R); and reducing means for reducing the timerate of displaying one of the first signal and the second signal whensaid determining determines that the first and second signals are nottime-synchronized relative to the clock C_(R) such that the one is timeadvanced relative to the remaining other of the first signal and thesecond signal, said reducing being directed to time-synchronizing saiddisplaying of the first and second signals on the display apparatus. 16.The system of claim 15, wherein the first signal is in an uncompressedformat as received by the receiving apparatus, wherein each timestamp ofthe first plurality of timestamps includes a time corresponding to alocation in the first signal at which said timestamp of the firstplurality of timestamps is positioned, wherein the second signal is inan uncompressed format as-received by the receiving apparatus, andwherein each timestamp of the second plurality of timestamps includes atime corresponding to a location in the second signal at which saidtimestamp of the second plurality of timestamps is positioned.
 17. Thesystem of claim 15, wherein the first signal is in an uncompressedformat as received by the receiving apparatus, wherein each timestamp ofthe first plurality of timestamps includes a time corresponding to alocation in a first uncompressed signal which was compressed at thesource to form the first signal, wherein said timestamp of the firstplurality of timestamps further includes an identification of saidlocation in the first uncompressed signal, wherein the second signal isin an uncompressed format as received by the receiving apparatus,wherein each timestamp of the second plurality of timestamps includes atime corresponding to a location in a second uncompressed signal whichwas compressed at the source to form the second signal, and wherein saidtimestamp of the second plurality of timestamps further includes anidentification of said location in the second uncompressed signal. 18.The system of claim 15, wherein the first signal is in an uncompressedformat as received by the receiving apparatus, wherein each timestamp ofthe first plurality of timestamps includes a time corresponding to alocation in the first signal at which said timestamp of the firstplurality of timestamps is positioned, wherein the second signal is inan uncompressed format as received by the receiving apparatus, whereineach timestamp of the second plurality of timestamps includes a timecorresponding to a location in a second-uncompressed signal which wascompressed at the source to form the second signal, and wherein saidtimestamp of the second plurality of timestamps further includes anidentification of said location in the second uncompressed signal. 19.The system of claim 15, wherein said reducing is effectuated by a delaycompensation controller of the receiving apparatus.
 20. The system ofclaim 15, wherein said reducing comprises introducing a time delay gapin the displaying of said one of the first signal and the second signal.21. The system of claim 15, wherein said reducing does not compriseintroducing a time delay gap in the displaying of said one of the firstsignal and the second signal.
 22. The system of claim 15, wherein thefirst modality differs from the second modality.
 23. The system of claim22, wherein the first modality is a video modality, and wherein thesecond modality is an audio modality.
 24. The system of claim 22,wherein the first modality is a video modality, and wherein the secondmodality is an text modality.
 25. The system of claim 22, wherein thefirst modality is a text modality, and wherein the second modality is anaudio modality.
 26. The system of claim 15, wherein said receiving meanscomprises means for receiving the first signal and the second signal onseparate channels.
 27. The system of claim 15, wherein said receivingmeans comprises means for receiving the first signal and the secondsignal as not multiplexed with each other.
 28. The system of claim 15,wherein said receiving means comprises means for receiving the firstsignal and the second signal as multiplexed but not time-synchronizedwith each other.